Asymptotic Peak utilisation in Heterogeneous Parallel CPU/GPU Pipelines: a Decentralised Queue Monitoring Strategy

Heterogeneous parallel computing has become an unavoidable consequence of the emergence of GeneralPurpose computing on graphics processing units (GPGPU). The characteristics of a Graphics Processing Unit (GPU)—including significant memory transfer latency and complex performance characteristics—demand new approaches to ensuring that all available computational resources are geared towards optimal utilisation. This paper considers the simple case of a divisible workload based on widely-used numerical linear algebra routines and considers the challenges that present themselves when an attempt is made to efficiently use all resources available with a view in balancing the CPU and GPU utilisation. We suggest a possible queue monitoring strategy that facilitates resource usage for applications that fit the pipeline parallel architectural pattern on heterogeneous multicore/multi-node CPU and GPU systems.